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Abstract 



We construct efficient robust truncated sequential estimators for the pointwise 
estimation problem in nonparametric autoregression models with smooth coeffi- 
cients. For Gaussian models we propose an adaptive procedure based on the con- 
pH ■ structed sequential estimators. The minimax nonadaptive and adaptive convergence 

C^ ' rates are established. It turns out that in this case these rates are the same as for 

regression models. 
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1 Introduction 

k> ' One of tlie standard linear models in the general theory of time series is the autoregressive 

model (see, for example, [1] and the references therein). Natural extensions for such models 
are nonparametric autoregressive models which are defined by 

Vk = S{^k)Vk-i^ik^ l<k<n, (1.1) 

where S{-) is unknown function, the design Xj^ = k/n and the noise (iCfc)i<fc<n ^^^ i-i-d. 
unobservable centered random variables, i.e. E^j^ = 0. 
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It should be noted that the varying coefficient principle is well known in the regression 
analysis. It permits the use of a more complex forms for regression coefficients and, there- 
fore, the models constructed via this method are more adequate for applications (see, for 
instance, [9], [23]). In this paper we consider the varying coefficient autoregressive models 
(11. ip . There is a number of papers which consider these models such as [7], [8] and [1]. In 
all these papers, the authors propose some asymptotic (as n — > oo) methods for different 
identification studies without considering optimal estimation issues. To our knowledge, 
for the ffist time the minimax estimation problem for the model (II. ip has been treated in 
[3] in the nonadaptive case, i.e. for the known regularity of the function S. Then, in p] it 
is proposed to use the sequential analysis method for the adaptive pointwise estimation 
problem in the case when the unknown Holder regularity is less than one, i.e when the 
function S is not differentiable. It turns out that it is only the sequential analysis method 
allows to construct an adaptive pointwise estimation procedure for the models (II. ip . That 
is why, in this paper, we study sequential estimation methods for the smooth function S. 
We consider the pointwise estimation at a fixed point z^ g]0; 1[ in the two cases: when the 
Holder regularity is known and when the Holder regularity is unknown, i.e. the adaptive 
estimation. In the ffist case we consider this problem in the robust setting, i.e. we assume 
that the distribution of the random variables (^j)j>i in (II. ip belongs to some functional 
class and we consider the estimation problem with respect to robust risks which have 
an additional supremum over all distributions from some fixed class. For nonparametric 
regression models such risks was introduced in [ll] for the pointwise estimation problem 
and in [16] for the quadratic risks. Later, for the quadratic risks the same approach was 
used in [20] for regression model in continuous time. Motivated by this facts, we consider 
the adaptive estimation problem for the Gaussian models (II. ip . More precisely, we assume 
that the function S belongs to a Holder class with some unknown regularity 1 < /3 < 2. 
Unfortunately, we can not use directly the sequential procedure from [2] for the adaptive 
estimation of such functions. Since to obtain an optimal rate for the function with /3 > 1 
we have to take into account the Taylor expansion of the function S at Zq of the order 1. 
To study the Taylor expansion for sequential procedures one needs to control the behavior 
of the stopping time. Indeed, one needs to keep the stopping time near of the number 
of observations. This can not be done by the procedure from [2] since one needs to use 
the unknown function S. In this paper we construct a sequential adaptive estimate for 
smooth functions and we find an adaptive minimax convergence rate for smooth func- 
tions. In Section |2l we present the standard notations used in sequel of the paper. We 
describe in detail the statement of the problem and main results in Section El In Section HJ 
we will study some properties of kernel estimators for the model (II. ip and in Section |5l 
we study properties of stopping time for constructed sequential procedure. Section |6] is 



devoted to the asymptotic upper bound and lower bound for the risk of the sequential 
kernel estimators. In Section [7] we illustrate the obtained results by numerical examples. 
Finally, we give the appendix which contains some technical results. 



2 Sequential procedures 

2.1 Main conditions 

We assume that in the model (11. ip the i.i.d. random variables {^k)i<k<n have a density p 
(with respect to the Lebesgue measure) from the functional class V^ defined as 

f P + OO f + OD 

'P^'-=-lp>0 '■ / p(x) dx = 1 , / xp{x)dx = 0, 



(2.1) 



V "^ — OO "^ — oo 



/+00 -1 /•+00 I 

x^ p{x) dx = 1 and sup ^ — -r / \x\'^'' p{x) dx < 1 ? 
-oo k>l ^ V /■■ J —oo J 



where <;" > 1 is some fixed parameter. Note that the (0, 1)-Gaussian density belongs to V. 
In the sequel we denote this density by Pq. It is clear that for any g > 

s; = sup Ep|eir<oo, (2.2) 

pen 

where E is the expectation with respect to the density p from V^. To obtain the stable 
(uniformly with respect to the function 5* ) model (11.11) . we assume that for some fixed 
< £ < 1 and L > the unknown function S belongs to the e - stability set 



e 



e,L 



|^gCi([0,1],R) : ll^ll <l-e and \\S\\ < l\ , (2.3) 



where C^[0, 1] is the Banach space of continuously differentiable [0, 1] — )■ M functions and 
1 1 5* 1 1 = supg^^^-^ l'S'(a;)|. Similarly to [H] and [3] we make use of the family of the weak 
stable local Holder classes at the point Zq 

Ui^\e,L,el) = {See,^, ■ \n,{zo,S)\ < e^} , (2.4) 

where 

^hi^o^ S)= {S{zq + uh) - S{z^)) du 

and /3 = 1 + a is the regularity parameter with < a < 1. Moreover, we assume that 
the weak Holder constant e* goes to zero, i.e. e* — )■ as n — )■ oo. Moreover, we define the 
corresponding strong stable local Holder class at the point z^ as 



n^P\e,L,L*) = {SeQ,^^ : n*{z,,S) < L*] , (2.5) 

where 



r2*(2;Q, S) = sup 



\Six)-Siz,) 



xe[o,i] I"*- ^ol 

We assume that the regularity (3 < (3 < (3, where /3 = 1 + a and /3 = 1 + a for some fixed 
parameters < a < a < 1. 

Remark 2.1. iVote that for the regression models the weak Holder class was introduced 
in m^ for the efficient pointwise estimation. It is clear that it is more large than usual 
one with the same Holder constant, i.e. 

It should be noted also that for diffusion processes the local weak Holder class was used 
in [T^ and [15^ for sequential and truncated sequential efficient pointwise estimation re- 
spectively. Moreover, in J77| / these sequential pintwise efficient estimators were used to 
construct adaptive efficient model selection procedures in L2 for diffusion processes. 

2.2 Nonadaptive procedure 

First, we study a nonadaptive estimation problem for the function S from the functional 
class (12. 4p of the known regularity /3 = 1 + a. As we will see later to construct an efficient 
sequential procedure we need to use S" as a procedure parameter. So we propose to use 
the first u observations for the auxiliary estimation of S{zq). In this step we use usual 
kernel estimate, i.e. 

^'^ = T^ ^^""^^ ^^-' y^ ' ^-^ = 5Z ^(".) yU ' (2.6) 

y i=i i=i 

where the kernel Q{-) is the indicator function of the interval [—1; 1]; u- = (a;- — ZQ)/h 
and h is some positive bandwidth. In the sequel for any < /c < m < n we set 

m 

\m= E ^K) ?/'-!' (2-7) 

j=k+l 

i.e. A^ = Aq^. It is clear that to estimate S{zq) on the basis of the kernel estimate with 
the kernel Q we can use the observations {yj)^. <j<k*y where 

k^ = [nZf^ — nh] + 1 and k* = [uZq + nh] . (2.8) 



Here [a] is the integral part of a number a. So for the first estimation we chose z/ as 

u = u{h,a) = k^ + L, (2.9) 

where 

L = L{h, a) = \enh] + 1 and ? = e'(/i, a) = h°'/ In n . 

Next, similarly to p], we use a some kernel sequential procedure based on the obser- 
vations {yj)u<j<n- To transform the kernel estimator in the linear function of observations 
we replace the number of observations n by the following stopping time 

TH = mi{k>u + l:A^^^>H}, (2.10) 

where {0} = n and the positive threshold H will be chosen as a positive random variable 
measurable with respect to the a - field {y^, . . . , y^^}. Therefore, we get 

^1 = h\H ^K)2/.-i^j + >'HQK„)yr^-iyr„ ^{a^,^>h). (2.11) 

\j=v+l J 

where the correcting coefficient k^ on the set {A^^ > H} is defined as 



and Xj^ = 1 on the set {A^^ < H}. 

Now, to obtain an efficient estimate we need to use the all n observations, i.e. asymp- 
totically for sufficiently large n the stopping time t^ ^ n. Similarly to [19] , one can show 
that Tfj ^ 7(5') iJ as iJ — 7- 00, where 

^{S) = 1 - S\z,) . (2.12) 

Therefore, to use asymptotically all observations we have to chose H as the number 
observations divided by 7(5"). But in our case we use k* — k^ observations to estimate 
S{zq), Therefore, to obtain optimal estimate we need to define H as {k* — k^)/'y{S). taking 
into account that k* — k^ ^ 2nh and that 7(5") is unknown we define the threshold H as 

H = H{h,a) = ^nh, 4> = ^{h,a) = ^^^T^ , (2.13) 

where S^^ is the projection of the estimator S^ in the interval ] — 1 + e,l — e[, i.e. 

Sj^ = min(max(S'^,, —l + e),l — e). 
In this paper we chose the bandwidth h in the following form 

h = h{/3) = {kJ^^ , (2.14) 



where the sequence k^ > such that 

K^ = hininf nK„ > and hm n k^ = (2-15) 

n—^oo n— >oo 

for any < 6 < 1. 

2.3 Adaptive procedure 

We will construct an adaptive minimax sequential estimation for the function 5* from the 

functional class (12 .Sp of the unknown regularity /3. To this end we will use the modification 

of the adaptive Lepskii method proposed in [2] based on the sequential estimators (12. lip . 

We set 

rf and iV(/3) = (rfJ^. (2.16) 

inn 

Moreover, we chose the bandwidth h in the form (I2.14p with k^ = l/d^, i.e. we set 

We define the grids on the intervals [f3 , f3] and [a , a] as 

k — k 

Pk = l^-^ (/3-/3) and a. = a H {a - a) (2.18) 

— 777- — 77T, 

for < k < m with m = [In d,^] + 1, and we set 

iV, = iV(/3,) and h = hi^,) . 
Replacing in (12. 9p and (I2.13P the parameters h and a we define 

i'k = ^iK "fc) and H^ = HCh^, aj • 
Now using these parameters in the estimators (12. 6p and (12.111) we set Sf^ = S'? (Zf^) and 

^k = max ( l^. - 5J - — ) , (2.19) 

where 

(— \ 1/2 
P^^ 
(2^+1) (2/3+1)^ 

In particular, ii f3 = 1 and /3 = 2 we get A^ = 4(2/15)^/^. We also define the optimal index 
as 

k = max lo<k<m:Uf^<—\. (2.20) 

The adaptive estimator is now defined as 

K,n = Sl and h = h-,. (2.21) 



Remark 2.2. It should be noted that in the difference from the usual adaptive pointwise 
estimation (see, for example, I21^ . ITBj . f^ and al.) the threshold A in fl2.19p does not 
depend on the parameters L > and L* > of the Holder class ( 12. 5p . 



3 Main results 

3.1 Robust efficient estimation 

The problem is to estimate the function S{-) at a fixed point Zq g]0, 1[, i.e. the value 
S{zq). For this problem we make use of the risk proposed in [3J. Namely, for any estimate 
S = Sj^{zq) (i.e. any measurable with respect to the observations {yk)i<k<n function) we 
define the following robust risk 

7^„(5„, S) = sup EsJS^iz,) - S{z,)\ , (3.1) 

pep, 

where E^^ is the expectation taken with respect to the distribution P^^ of the vector 
(yi, •••,y„(,) in (11.11) corresponding to the function S and the density p from V^. 

With the help of the function 7(5) defined in (12.121) . we describe the sharp lower 
bound for the minimax risks with the normalizing coefficient 



(^„ = n2/+i. (3.2) 



Theorem 3.1. For any < £ < 1 



hm^^^ inf sup ^-'/'{S) v^„7^„(5„, S) > E|r/| , (3.3) 

S SGU(fHe,L,el) 

where rj is a Gaussian random variable with the parameters (0, 1/2). 



Now we give the upper bound for the minimax risk of the sequential kernel estimator 
defined in f l2lTll . 

Theorem 3.2. The estimator (12. 6 p with the parameters (I2.13P - (I2.14p and k^ = n~^ 
satisfies the following inequality 

lim„^^ sup -f'^^^iS) ^„7^„(5„„, S) < E\r]\ , 

5eW('3){e,L,£*) 

where rj is a Gaussian random variable with the parameters (0, 1/2). 

Remark 3.3. Theorems \3.1\ and \3.1\ imply that the estimator (12. 6p . with the parameters 
(I2.14P is asymptotically robust efficient with respect to class V . 



3.2 Adaptive estimation 

Now we consider the Gaussian model (11.11) . i.e. assume that the random variables (^j)j>i 
are J\f{0,l). The problem is to estimate the function S* at a fixed point Zq g]0,1[, i.e. 
the value S{zq). For any estimate S^ of S{zq) (i.e. any measurable with respect to the 
observations {yk)i<k<n function), we define the adaptive risk for the functions 5* from 

n^^'^{e,L,L*) as 

7^,,J^„)= sup_ sup NiP)Es\S^-Siz,)\, (3.4) 

where N{f3) is defined in (I2.16p . E^ = E^^ is the expectation taken with respect to the 
distribution P^ = P^.^ . 

First we give the lower bound for the minimax risk. We show that with the convergence 
rate N{P) the lower bound for the minimax risk is strictly positive. 

Now we give the upper bound for the minimax risk of the sequential kernel estimator 
defined in fl2Jl]) . 

First we give the lower bound for the minimax risk. We show that with the convergence 
rate N{I3) the lower bound for the minimax risk is strictly positive. 

Theorem 3.4. There exists L* > such that for all L* > L* the risk (13.41) admits the 
following lower bound: 

liminf inf TZ^ „(5'„) > , 

where the infimum is taken over all estimators Sn- 

The proof of this theorem is given in p] . 

To obtain an upper bound for the adaptive risk (13. 4p of the procedure (12.211) we need 
to study the family (5'*)^<^<jj. 

Theorem 3.5. The sequential procedure (12. lip with the bandwidth h defined in (I2.14p for 
K^ = In n/n satisfies the following property 

limsup sup (Tn{h))~^ sup sup Eg^^lS^^ — S{zq))\ < oo 

n-s>oo a<a<a S&'H'-l^\e,L,L*) P&V^ 

where Tn{h) = h'^ + {nh)~^/'^ . 

Using this theorem we can establish the minimax property for the procedure (13. 4p . 
Theorem 3.6. The estimation procedure (13. 4p satisfies the following asymptotic property 

limsup7^ (5 )<cx). (3.5) 



Remark 3.7. Theorem \3.4\ gives the lower bound for the adaptive risk, i.e. the convergence 
rate N{I3) is best for the adapted risk. Moreover, by Theorem \3.(A the adaptive estimates 
(I2.2ip possesses this convergence rate. In this case, this estimates is called optimal in 
sense of the adaptive risk (13 .4^ 

4 Properties of S^ 

We start with studying the properties of the estimate (12.131) . To this end for any g > 1 
we set 

^; = ^'"ILj^;^^" (b; (-;«; + -i, + 1) + 2(1 + l^ r^ j , (4.1) 

where 

IVA ^ ^, 18 V^^ 



r; = 2-^(ll/or + s;(i)') and b^ 



g (g- 1)9/2' 
Now we obtain a non asymptotic upper bound for the tail probabihty for the deviation 

\ = S^- S{z,) . (4.2) 

Proposition 4.1. For any q > 1, h > and a > Lh 



sup sup Pcjp (|AJ > a) < M^g (Inn)" /i^^-")" ' ^'' 



M2 

566,, pen - ^ - W^Lhy]'^/^ 

where M-, „ = 2" g* and M. = 2" b* s* r* . 



Proof. First, we write the estimation error as follows 



K = B, + ^c, 



where C = Ej=i Qi'^j)yj-i^j and 

Note that |-B,^| < -^/i for any S & Q^j^^. Putting v = l/2 we can write 

(|AJ > a) = P^p (|AJ > a, A, < t;) + P^.p (|AJ > a , A, > t;) 

< P5,p (A <v) + Ps,p (^Lh+^-^> a, A, > v^ 

< Ps,p (A. <v) + Ps,p (7^ > « - ^^' ^. > ^) ■ (4-3) 



Ps,p \\^u\ 



Now, for any R — > R function / and numbers < /c < ?7i — 1 we set 

^>^.M) = ^^t nu,)yU-~t f(-,)- (4.4) 

j=k+l j=k+l 

Using this function we can estimate the first term on the left-hand side of (14.31) as 

= p.. {e,,JQ) < ~) 

< Ps^p {K.AQ)\ > ?/2) < (IJ EsJe>,^JQ)\' . 
Therefore, using here Lemma [8.31 we get 

where the coefficient g* is defined in ( 14. ip . The last term on the right-hand side of ( 14. 3p 
can be estimated as 

P5,p (^ICI >a-Lh,A^>v^< Ps,pi\U > ^(« - Lh)) 

1 



< 



Now in view of the Burkholder inequality, it comes 



E5,picr. 



<? / \ 9/2 



E5,picr = E,,, Y. ^(".) ^.-1 ^. ^ K ^s,, E ^'(".) yU ^] 



.2 e2 



fc.+. ^ "/' 



KE.. E ^;-i^; 



r2 



and after applying the Holder inequality, we obtain 

k^+i 
o,p\^i'\ — q / J ijp^j — i-^j — g g q 

Therefore, 

/ 1 \ 29b*s*r* 



Hence Proposition 14.11 D 



10 



Proposition 4.2. Let the bandwidth h be defined by the conditions fl2.14p - fl2.15p . Then, 
for all m > 1 and < 6 < 1 



lim sup h ™ sup sup P^^ ( |Aj^| > 5ej = . 

Proof. By applying Proposition 14. II for a = (5?, we obtain that for sufficiently large n > 1, 
for which Se > 2Lh, and for any m > 1 and q > m/{l — a) 



/i-™P5_p(|AJ >67)< /i-"Mi Jlnn)«/i(i-")'? 



^2,,h-^ 



{L{5e-Lh) 



2\Q/2 



(l-^)g-m 29 M. h~im+q/2) 



(1 — tK)g — 771 



< Ml Jlnn)'?K^ 2/j+i + 



29M2„ (Inn) 39/2 ,(i-t./2) 



(59(K„n)9/ 



2 n 



K 2/3 + 1 



Taking into account here the conditions fl2.15p we come to Proposition 14.21 D 



5 Properties of stopping time r^ 

First we need to study some asymptotic properties of the term (12. 7p . 



Proposition 5.1. Assume that the threshold H is chosen in the form f l2.13p and the 
bandwidth h satisfies the conditions fl2.14p - (12.151) . Then for any m > 1 

lim sup sup /i~™ sup sup Ps,pi\,n < H) < oo . 



Proof. Using the definition of H in fl2.13p we obtain 






".■^ %-i < ^ 



H 

nh 



j=u+l 



Note that 

Taking into account that e^ < 7(6*) < 1, we obtain 



_k* -k^-L 6 + 2 

•^ nh ~ nh 



1 



7(^J 7(5) 
11 



<1|A| 



(5.1) 



This yields 



vj=u+l 



<P5,p(^.,.-(Q)<-2?n + J + ^5?+ ^ 



e'^nh 



+ P5,P lAJx^l • 



Therefore for 6 < e'^/8 and sufficiently large n > 1 we obtain that 

Ps,p {K,n <H)< Ps,p {\OuMQ)\ > ^n/2) + Ps,p {\\\ > S^) ■ 



Lemma 18.31 and Proposition 14.21 imply Proposition 15.11 D 

Now for any weighted sequence ('U^j)j>i we set 

n 

^n= Yl '^(^j) wj y'^._-^ + (1 - Xh) w^^ QK^) yl^_-^ 



(5.2) 



Proposition 5.2. Assume that the threshold H is chosen in the form fl2.13p and the 
bandwidth h satisfies the conditions f l2.14p - f l2.15p . Moreover, let ("U^jOjm be a sequence 
bounded by a constant w* , i.e. sup.^-^ \Wj\ < w* . Th 



en 



r L-a Eg |Z„| 

hm sup sup h sup sup — —, = (J . 



n— !>oo a<a<a SsO^ £, P^V^ 



nh 



(5.3) 



Proof. It is clear that Z„ = if A^„ < if , and on the set {A^^ > H} this term can be 
estimated as 

\j=v+l J 

i.e. \Z^\ < w*{Ai^^ — H)^, where (x)_,_ = max(0,x). Therefore, 



nh 



- * 

< w 



E" ^K)^?-i 



< \0UQ)\ 



nh 



e;=,+i Qi^j 



^{S)nh 



Taking into account that YT- .i Q{Uj) = k* — k^ — l < 2nh we obtain 



12 



nh 



<EcJ^...(Q)|+2 



7(^) 7(5J 



2 ^ 



<EcJ^...(Q)| + -7E5,p A, (2:0) 



+ -e. 



Moreover, note that 



E 



S,p 



A.(^o) 



E 



S,P 



+ E 



S,p 



A.(^o: 



A.(^o) 



<e + 2P 



S,p 



-{\A^{zo)\<7} 
■{\A^{zo)\>I} 

> e 



A„,>o 



Therefore.Lemma 18.31 and Proposition ITT] imply immediately (15.31) . D 



6 Proofs 



6.1 Proof of Theorem 13.11 

First, similarly to the proof of Theorem 2.1 from [3] we choose the corresponding para- 
metric functional family S^^{-) in the following form 



^uA^) 



^ 



u I X-Zo 



(6.1) 



with the function V^ defined as 



V,ix) = 5~' I Qsiu)gi^^^]du. 



where Qs{u) = l||„|<i_25} + 21|i_2<5<|«|<i-<5} with < 6 < 1/4 and g is some even nonneg- 
ative infinitely differentiable function such that g{z) = for \z\ > 1 and J g{z) dz = 1. 
One can show (see [13]) that for any 6 > and < 6 < 1/4 there exists n^ = n^{h, L, (5) > 
such that for all |ti| < 6 and n > n^ 

Therefore, in this case for any n > n^ 

^„ sup 7-'/'(5)7^„(5„,5)> sup 7-'/'(^)E5,,„V'n(S„,5) 

1 fb 



13 



where '-^^{n^h) = inf|„|<j 7 ^^'^{S^^). The definitions fl2.12p and (16 .ip imply that for any 



b>0 



hm sup 17(5^,5) - 1| = 0. 

"-^°o|«|<b 



Therefore, by the same way as in the proof of Theorem 2.1 from [3] we obtain that for 
any 6 > and < 5 < 1/4 

lim^^^inf sup 7-'/'(^)^n7^JS„,^)>/(6,a,), (6.2) 

where 

^,, , max(l,6-v^) a, /"^ _,.^ 

with cr^ = / y^iu) du. It is easy to check that a^ — )■ 2 as 5 — )• 0. Limiting 6 — )■ 00 and 
(5 — )• in (16. 2p yield the inequality (13. 3 p . Hence Theorem 13.11 D 

6.2 Proof of Theorem [372] 

First we set 

% = l{-«7^.} + ^^ l{r^=.} • (6-3) 

Then taking this into account we can represent the estimate error as 

1 



SI - S{Z,) = -S{Z,) 1(^^^<^) + h^ BM 1(A^,„>H) + ^ Cnih) 1(A_>/.) 

where 

Bnih) = j^ 

and 

CnW = -^ — 



(6.4) 



First we study the term B^{h). To this end we introduce 

j=u+l 

It is clear that for any S from U^^\e, L, e*) 

sup \b*\ < L . 

Therefore, using Proposition 15.21 for the sequence (15. 2p with w, = b* we obtain 

hmsup sup n sup sup = U. (6.5) 

14 



Moreover, putting 



A{u)=Q{u) 



S{zq + hu) - S{Zf^ 
h 



we obtain 



B* 



1 " 1 '^ 



Using the definition of ^^{z^^ S) in (12. 4p we can represent the first term as 

Y, fiiu,)Au^ = -^n,{z,, S)- f,iu)du +J2 (/iK) - /i(«)) d^ 

fc=i "^"fc* fc=fc. "^"j-i 

Note now that for any S from lA^^\e^ L, e*) 

sup \fi{u)\ < L and sup |/;^('u)|<L, 

-!<«<! -l<n<l 



(6.6) 



and, therefore. 



fiiu)du 



k* 



< —- and 
nh 



k* 



k=k^ "^"i-i 



< 



2L 



The last bounds imply immediately 



limsup sup h ° sup 

n-s>oo a<a<a S£U'-'^'>(e,L,e'') 

Taking into account Lemma 18.31 in (16. 6p we get 



E /iK)^". 



k=l 



0. 



limsup sup h " sup sup 

n^oo a<a<a S€U'-l^'l{e,L,e*) p&V^ 



nh 



Therefore, in view of (16.51) 



limsup sup sup sup E^.^ |i?^(/i) =0- 

n->-oo a<ct<a S&U^^\£,L,e*) peV^ 



To study the last term in (16. 3p note that the definition of the stopping time in (12.101) 
implies 

sup sup sup supE5p|C„(/i)p < 1. (6.7) 

n>u+l h^<h<h* 5ee^^ p&V^ 

Therefore, in view of Lemma [8.51 we obtain 



lim sup sup 



E5,plC(/^)|l(^_>H)-E|C 



where Coo ~ Ar(0, 1), i.e. E|C 
bound (15. ip we get 



2/7r. Moreover, in view of Proposition 14.21 and the 



lim sup sup E^.^ 



' r. 



l{S)H 2 
From this and Proposition 15.11 it follows Theorem 13.21 D 



0. 
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6.3 Proof of Theorem [331 

First, note that the representation (16 ■4p imphes 



+ E 



S,P 



lUMi 



H 



(^.,„>^) 



Let us show, that 



hmsup sup sup sup E^p I !?„(/;,) I < oo 

i<a<a S&'H'-'^'>{e,L,L*) p&V 



n—>-oo a< 



Indeed, setting 



we can represent B^{h) as 



>S(x .) - Sjz^) . 



BM = ^S{z,)B^ + ^B^, 



where 

Using now Proposition 15.21 for the sequence ( 15. 2p with Wj = Uj we obtain that 



Bn= Yl ^jQi^j)^jy^j-i and S„= ^ kjQ{uj)i^jy^._^. 



hm sup sup /i sup sup — - — -, = U , 



n— ^-oo a<a<a SgS^ j^ P&'Pr 



nh 



where 






We can represent this term as 



1 



fc^+t 



where Qi{u) = Q{u)u. Moreover, taking into account here, that 



u. 



Yl "i^^i 



U 






< 



nh 



(6. 



(6.9) 



(6.10) 



(6.11) 
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and 






nh ' 



we obtain 



;?7-E.,,lKl<E,,,IM^i)l + if;^+^)- 



nh 



e^ \ nh 



Now Lemma [8.31 yields 



"p It/ I 
limsup sup h sup sup —, = U 



71— >oo a<a<a 5e0^ ^ pST'^ 



nh 



and, therefore, 



lim sup sup /i sup sup ; = U . 

nh 



(6.12) 

To estimate the last term in (]6.1ip note, that for any function 5* from Ti^^^e, L, L*) and 
for k^ < j < k* the coefficients (16.101) can be estimated as 



1^,1 



S{zq + hu) — S{zq) ) dn 



<L|m,|/i"< L*/i" 



Therefore, 



limsup sup h " sup sup — Egp|_B„| < oo . 



nh 



Now the property (16.121) implies the inequality (16. 9p . Therefore, using Proposition 15.11 and 
the inequahty (16. 7p in (16. 8p , we come to Theorem 13.21 D 



6.4 Proof of Theorem 13.61 



First of all, note that the coefficient defined in (I2.13P will be more than one for sufficient 
large n for which ?< 1/2. So, using the representation (16. 4p . we get for any 1 < j <m 



\S^ - S{z,)\ < Ir,, + {h^f \BSh,)\ + ^= \ah)\ 



nh- 



where V- = {A^^„(/ij) < H-}, C^hj) = Cn(^) l(A,_„{h^)>//^.) 
and Cn(^) ^^6 defined in (16. 4p . We set 



(6.13) 



jj- -), the random functions B^{h) 
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This means that 



< /3 - A- < 



/3-/3 



m 

Therefore, taking into account the definition of m in ( 12.18^ . we obtain that for any fixed 

integer — < oo < / < oo 



< hminf,. 



< hminf^ 



< hm sup , ^ _ ^T — < C)0 



A^. 



.+i 



N, 



+1 



HP) , ,. KP) 

< hm sup , _ < cxo 



(6.14) 



h 



■^+1 



K 



,+i 



These inequahties and Theorem 13.51 imply 



hmsup sup N{f3) sup EigZu{iQ,ZQ) < oo 

n-s-oD I3<l3<i3 S&H'-'^^{e,L,L*) 



(6.15) 



where ^(io,^;^) = \S^^_^ - S{zq)\ + |^.^ - S{zq)\ + 1^^^+^ - S{z^^)\. Now considering the 
estimator S„ „ , one has 



l-^. „ - ^(^o)! <h + h + ^(*o. ^( 



o; ' 



(6.16) 



where /^ = |5„„ - 5(2;o)|l|^>.^+2} and I^ = |5„„ - 5(2o)|l{fc<io-2}- We start with the 
term I^. We have 



\^a,n ^{^o)\'^{k>ifj+2} ^ \^a,n "^Jq ' "'"{^>*0+2} ~'~ I *o "^(^o) I l{fe>io+2} • 



Moreover, 



- ]^^i^>^o+2} + aT - iV(^ + aT • 
The inequahties (I6.14p -( l6.15p imply immediately 

limsup sup N{(3) sup ^sh < oo . 

n-5-oo I3<I3<]3 S&'H''l^\e,L,L*) 

Now we study the term Jg. From (I6.13P it follows that 



Therefore, 



nhv 



in-2 



Esh < mIliS) + i;{S) Y, h^, + ^niS) 

j=0 



(6.17) 
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where J*(^) = maxo<K^ P5 O^i), ^iS) = maxo<;<„ EslB^ih^)] and 






Note now, that for any < j < ig and for sufficiently large n (for which Ind^ > 771/2) we 
get _ 

N{li)hP. < e-/^*(^o-^-) , p* = (^^(^-§ 

^^ ^ - (2/3 + 1) (2/3+1)' 

i.e. 



NiP)J2M< ' 



i=o 
So, Proposition 15.11 and the inequality (16. 9 p yield 

limsup N{P) sup m/;(5) + /*(5) ^1 ^^ < °^ • 

Let us consider now the last term (I6.17P . To this end note that for < j < i^ — 2 

{k = j} C {cJ^^^ > A/iV^.+J C u^;j{|5, - S{z,)\ > A/iVj . 
Therefore, 

^^(^) ^ 7^ E ^ E E5 ic(^-)i i{i5,-5(.,)i>AM} • 



j,=0 -W^j «=0 

Taking into account that hj_^^/hj < e, we can rewrite the last inequality as 



(6.18) 



Now, taking into account the inequality (16. 7p . we get 



+ E5 |C*| i{ic*i>v^n7nA+ 

where A^^ = A/2 and (* = maX]^<^.<^ lCn(^j)l- ^^ view of the Holder and Chebyshev 
inequalities and making use of the upper bound ( 18. lip we obtain 

E5 IC(^-)I l{/.f|B„(/.,)|>A,/JVJ < JsTa W^ ) ■ 
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where the term I^iS) is defined in fl6.17p . Using these bounds in fl6.18p we get 



^n(^) < ^7^ \in(s) + "^"^^ y;-^^ t; 



'0 

''-{lC*l>AiVI^} 



6 ?77/ 

+ ^=E5lC|liic*i>A.V5r;il' (6-19) 

nhn 




where 

T- 1 g («/,»)"* fiV,/,")"^ „. and „• = t f ^V* ■ 

^ j=l 1=0 \ J J J 

Let us estimate the term T* . To this end note, that for any < j < ig and for sufficiently 
large n (for which In d^ > m/2) 



(iV^./^;)'^' = exp|^|^^lnrf„|<e-^I('o-.) with /3; 



4(2/3^. + !) "J - '^i 8(2/3 + 1) 

and 

- \ 1/2 r /3 . - a 1 1 ./Z" 

AT./i") = exp <^ -^^- ln< ^ < ^^ " 



^ ^V ^ 12(2/3, + 1) "j -./^ iV(/3) 



where h{P) is defined in (I2.17p . Similarly for any < / < j < Zq we get 

nMX^' f 3(/3,-/3^.)(2/3 + l) \ 3(^-/3) 



N^h^^ .^.-.. , ^..-., 



This means that the sequence ('U*)j>i is bounded, i.e. 



sup V . < —^ 

j>i ' ~ e"' - 1 



Therefore, 

lim N{l3) T* = . (6.20) 

n— >oo 

The last term in (16.191) can be estimated through Lemma FS. 101 i.e. 



r+oo 

< 2mAiyi^e-5^i^^" + 2m / e^'^^dz. 

'J \^ \/\r\ n. 



' X^ vlnn 
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Therefore, for sufficiently large n (when A^ \/\nn > 1) we get that 

E5 C l{c>K vT^} ^ 2m (Ai v/i^ + 4) n-'V' . (6.21) 

Now the definition of the parameter A in (I2.19P yields 

limsup - sup Es|C|l||^.|>j,^i^|=0. 

Hence Theorem 13. 6[ D 

7 Numerical examples 

We illustrate the obtained results by the following simulation which is established using . 

7.1 Nonadaptive estimation 

In this section we illustrate the results obtained in the case of nonadaptive estimation 
The purpose is to estimate, at a given point Zq, the function 5* defined over [0; 1] by 

(7.1) 











S{x) = {x - 


-Zo)\^-^o" 




^0 = 


= 1/^2 and 


a = 


0,3. 


Taking into account that for this function 










^.(^0: 


,S) = 




obtain that for 


any 

















< e 


r 1 \^ 


L>/3 and 


e* >0 

n 



(7.2) 

the function (17. ip belongs to the class U^\e, L, e*) with /3 = 1, 3. 

The numerical results approximate the asymptotic risk of a estimators defined in (12. lip 
used due to the calculation of an expectation (it performs an average for M = 30000 
simulations) and the finite number of observations n. Here we calculate for the estimator 
the quantity 



k=l 



For the standard Gaussian random variables (^~),>i in (II. ip . and by varying the num- 
ber of observations n, we obtain different risks listed in the following table: we obtain: 
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n 


1000 


5000 


10000 


20000 


Rn 


0.034 


0.021 


0.017 


0.012 



For random variables (^,)j>i reduced from uniform random variables on [—1, 1], we 
obtain : 



n 


1000 


5000 


10000 


20000 


Rn 


0.038 


0.022 


0.018 


0.014 



For random variables (^j)j>i centered and reduced from exponential random variables 
with parameter 1, we obtain : 



n 


1000 


5000 


10000 


20000 


Rn 


0.028 


0.016 


0.012 


0.010 



7.2 Numerical result for non sequential kernel estimator 

Now we give the numerical results for the kernel estimator defined as 



'S'n(^o) 



^ Q{uk)yk-iyk- 



ELiQK)i/Li fit 

For the standard Gaussian random variables (CiOjm i^^ (II. IP - and by varying the num- 
ber of observations n, we obtain different risks listed in the following table: 



n 


1000 


5000 


10000 


20000 


Rn 


0.046 


0.026 


0.02 


0.015 



7.3 Adaptive estimation 

In this case we estimate the function f l7.ip for z^ = l/-\/2 and a = 0, 7. Obviously, that 
this function belongs to class 'H^'^\e,L, L*) with /3 = 1,7, L* = 1 and for any e and L 
satisfying the conditions (17. 2p . 

In the adaptive estimation we take the lower regularity /3 = 1.6 and the higher regu- 
larity ^ = 1.8. 

We model the sequential adaptive procedure S^^^ = 5? defined in (12.2 ip . Numeri- 
cal results approximate the asymptotic risk for this procedure by the calculation of an 
expectation via M = 30000 simulations. 



R„ 



1 ^ ^ 

— Y\S'' -S(Zr.)\ 

/l/f / ■/ ' a,n ^ U/ 1 



fc=l 



By varying the number of observations n, we obtain different risks listed in the follow- 
ing table: 
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n 


1000 


5000 


10000 


20000 


R-a.n 


0.021 


0.013 


0.009 


0.007 



8 Appendix 



8.1 Concentration properties of the process ( 11.11 ) 
In this section, we study the deviation (14. 4p for the model (II. ip . 



Lemma 8.1. For any q > 1 and < £ < 1 the random variables y^ in (II. ip satisfy the 
following inequality: 

sup sup sup sup Ec.plyfcl" < r* (8.1) 

n>l 0<fc<n See^ i p£V^ 

where r* is defined in (14. ip . 



Proof. From (II. ip we obtain that for any k > 1 



j=l Z=j+1 



Therefore, for S G Q^ ^^ and 1 < k < n, 






Moreover, the Holder inequality gives 



<?-! 



Ed 



^)'"^' 1^,1 I < 



E(i-^)'"M E(i-^)'"'K 



k-j If \g 






.i=i 



(i)'"(i:(i-.)-i?,i') 



Thus, taking into account the definition (12. 2p we get for any p & V^ 



BsJtd-er'liA <(!)' 



Hence Lemma [8. II D 

Now we need the following Burkholder inequality from [2^ . 
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Lemma 8.2. Let {Mj^)i^f^^^ be a martingale. Then for any q > 1 

E |M„|» < b;E J](M_, - M_,_i)' I . (8.2) 

where the coefficient h* is defined in (14.11) . 

Now we study the deviation ( ]4.4p . 

Lemma 8.3. Let / &e a M — t- M function twice continuously differentiahle in [—1,1]. 
Assume also that the bandwidth h satisfies the condition (12.141) - (I2.15p . Then for any 
R> and q > 1 

limsup sup sup_sup sup — — - sup sup Eg^ |^;^,„(/)|'' < ^* (8.3) 

n^oo k^<k<m<k'' I3<I3<I3 R>0\\f\\i<R l-f^"'J -^ee^^^ peV^ 

where \\f\\^ = \\f\\ + ||/|| and g* is defined in ( KT\i . 
Proof. First of all, note that 

m 

Yl f^^j^y]^! = ^k,m + %,m . (8-4) 

j=k+l 



where T^,^ = E™ ^+1 fi'^j)y^ and 

m 

(^k,m= Yl (fi'^j) - f('^j-i))y]-i + f(^k)yl- fiujyl^. 
j=k+i 

Moreover, from the model (11.11) we find 

m 

(1 - '5^(^o))^fc,m = ^fe,m + «fc,m + Yl •^K'^ 

j=k+l 

where M,,^ = EJl.+i (25(x,) y,_i ^, + ^J - 1) /(«,) and 

m 

H,m= Y f(^j)^^(^j)y^j-l- ^^(^o)'^k,m- (8-5) 

j=k+l 



Then we can write Q^mif) as follow 



1 /-, -, X «fc,m 



0k,mif) - ^^^^g^ {Mk,m + a^J + ^^ 
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and 

39-1 f f\M.\y f\a.jy fWJW 

where k = (1 + k^)/k^. Now we note, that in view of the first condition in fl2.15p for 
sufficient large n 



nK^> K, 



where k = (1 + h^)/k^. Therefore, for sufficiently large n we get 

-L<(nf/('^+') h<Kh, (8.7) 

nh 

Furthermore, note that {M,^-)^^-^^ is a martingale. So, by applying the Burkholder 
inequality (18. 2 p and, taking into account that k* — k^ < 2nh, we get 



g/2 



^ ujb^ E E5. (2 5(x,) y,_, e, + e; - 1) 



, , ., (r* s* + s! + 1) < 4«b*K'? W (r* s* + s! + l) h'^ , 

where the coefficients r* and s* are given in (18. ip and (12. 2p . Note that the term (18. 5 p can 
be rewritten as 

(m 
j=k+i 

m 

j=k+i 

We recall, that the function / and its derivative / are bounded by R. Therefore, taking 
into account that for all S G ^^,1 ^^nd k^ < j < k* the deviation \S{Xj) — S{zq)\ < 
L\x- — Zq\ < Lh, we obtain 



fc,m 



<^[{^,^^^)±J^yl^yl) 



<kr \{L + i)h J2 y] + yl + yl 

j=k+i 
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Therefore, 



<(4ij(i + L))«ty!«. 

Similarly, 

sup sup EsJ^\a,jY <32''-'K'^R'>r* h'^. 

Then, taking this into account in (18. 6p we obtain the upper bound (18. 3p . Hence Lemma [8^ 
D 

8.2 Uniform limit theorem 

In this section we study the following sequence 

CM = Uh) \a^^^>h) , (8.8) 

where (nW defined by (16. 4p . the bandwidth h is defined (I2.14p and the threshold H is 
given in (I2.13p . We will make use of the following result. 

Lemma 8.4. (cf. fTlf . p. 90-91) Let < 5 < 1 and r > 0. Assume that (m^)^^^ is a 
martingale difference with respect to the filtration {J^k)k>i such that 

oo 

|m^| < (5r^/^ and ^ E (m^| J'^_-^) > r . 
fc=i 

Let 

r = inf JA;>l:^E(m2|j;._J> 

There exists a function p : (0, +oo) — > [0, 2] not depending on distribution of the martin- 
gale difference, such that lim^._5^Q p(x) = and 



sup 
xeR. 



fc=i 



<pW 



where $ is the standard normal distribution function. 
Lemma 8.5. The sequence (16.41) satisfies the following limiting property: 
C„ =^ C ~ -^(0, 1) uniformly in p eV^ and 5 G 6^ ^ . 
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Proof. First for some < 5 < 1 we set 

where y^ = yj'i-{\y^\<5^ h^/^}, H = Epi^H and ^^ = ^^ l{|5^.|<5-i} - E^^i l^|^^|<5-i}. It is 
clear that the sequence (m^,_^j)j>Q is a martingale difference with respect to {Qj)jyQ, where 
Qj is a - field generated by the observations {y-^ , • ■ ■ , y^+j}- Now we set 



1 ^" 



H 

m. 






where f^ = inf < /c > 1 : X] -i -^ (m^ , .|^j_i) >H>. Note, that f^j = Tjj on the set 
{^un — ^} f*^^ ^'^y H > 0. Lemma [8.41 implies that (fj goes in distribution to A^(0, 1) 
uniformly in p ^V^ and S E Q^ i^ as 6 -^ 0. Now we set 



Using Lemma 18.11 through the Chebyshev inequality we obtain that 

/- ■\ {k* — v)r* 
sup sup 'P q„ \VL'^ ) < )■ as n — )■ oo . 



Moreover, note that on the set f2„ fl {A^^ > H} 

C - a = A, + A^ , (8.9) 



^XV2 



where 



V -o , 1 y ri . . , 

and § = -Cj - ^j = ^j l{|c^.|>5-i} - E6 l{|ai>5-n- ^°*^' *^^* 

E,,, (A^l^o) < 5'E,,, ( E (1 - ^^■)'l^o ) < 5' • 
Moreover, taking into account that y"^ < y^, we get 



■:2 



<^{H + 6^H)=E^^ll ' 
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Taking into account here, that 



we obtain 



hm sup Ep ^^ =0 and hm sup | E^ .^^^ — 1 1 = 



hm sup sup max (E^.^ A^ , E^^ A^) = . 



Therefore, Proposition 15.11 and the representation (18.91) yield for any /i > 
hm hmsup sup sup P^.^ \ Cn~ Ch > /w) = . 

Hence Lemma [8.51 D 

8.3 Properties of Cn(^) 
Lemma 8.6. For all z > 2 



sup sup sup Ps(Cnih)>z)<2e''^/^. (8.10) 



n>l h>0 5eC[0,l] 

The proof of this Lemma is the same as Lemma A. 5 from |2]. 
Using this lemma we can obtain that for any q > 2 

sup sup sup E^ICJ/i)!" </"!, (8.11) 

n>l h>0 SeC[0,l] 

where /i« = 2« + 2g J^ f^'^ e-*'/Mt. 
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